Computationally Efficient M-Estimation of Log-Linear Structure Models
نویسندگان
چکیده
We describe a new loss function, due to Jeon and Lin (2006), for estimating structured log-linear models on arbitrary features. The loss function can be seen as a (generative) alternative to maximum likelihood estimation with an interesting information-theoretic interpretation, and it is statistically consistent. It is substantially faster than maximum (conditional) likelihood estimation of conditional random fields (Lafferty et al., 2001; an order of magnitude or more). We compare its performance and training time to an HMM, a CRF, an MEMM, and pseudolikelihood on a shallow parsing task. These experiments help tease apart the contributions of rich features and discriminative training, which are shown to be more than additive.
منابع مشابه
Estimation of portfolio efficient frontier by different measures of risk via DEA
In this paper, linear Data Envelopment Analysis models are used to estimate Markowitz efficient frontier. Conventional DEA models assume non-negative values for inputs and outputs. however, variance is the only variable in these models that takes non-negative values. Therefore, negative data models which the risk of the assets had been used as an input and expected return was the output are uti...
متن کاملESTIMATING THE PARAMETERS OF A FUZZY LINEAR REGRESSION MODEL
Fuzzy linear regression models are used to obtain an appropriate linear relation between a dependent variable and several independent variables in a fuzzy environment. Several methods for evaluating fuzzy coefficients in linear regression models have been proposed. The first attempts at estimating the parameters of a fuzzy regression model used mathematical programming methods. In this the...
متن کاملApplication of Recursive Least Squares to Efficient Blunder Detection in Linear Models
In many geodetic applications a large number of observations are being measured to estimate the unknown parameters. The unbiasedness property of the estimated parameters is only ensured if there is no bias (e.g. systematic effect) or falsifying observations, which are also known as outliers. One of the most important steps towards obtaining a coherent analysis for the parameter estimation is th...
متن کاملContrastive Estimation: Training Log-Linear Models on Unlabeled Data
Conditional random fields (Lafferty et al., 2001) are quite effective at sequence labeling tasks like shallow parsing (Sha and Pereira, 2003) and namedentity extraction (McCallum and Li, 2003). CRFs are log-linear, allowing the incorporation of arbitrary features into the model. To train on unlabeled data, we require unsupervised estimation methods for log-linear models; few exist. We describe ...
متن کاملLeast-squares Probabilistic Classifier: a Computationally Efficient Alternative to Kernel Logistic Regression
The least-squares probabilistic classifier (LSPC) is a computationally efficient alternative to kernel logistic regression (KLR). A key idea for the speedup is that, unlike KLR that uses maximum likelihood estimation for a log-linear model, LSPC uses least-squares estimation for a linear model. This allows us to obtain a global solution analytically in a classwise manner. In exchange for the sp...
متن کامل